Flow line recognition system

ABSTRACT

According to one embodiment, the flow line recognition system includes a first device, a first recording unit, a second device, a second recording unit and a generation unit. The first device detects a position of a moving object in a monitoring area by laser scanning. The second device detects a position of the moving object in a specific area, which is a part of the monitoring area, more accurately than the first device. The generation unit generates flow line information indicative of a path of the moving object moving in the monitoring area, based on the moving object information for the monitoring area recorded in the first recording unit, and the moving object information for the specific area recorded in the second recording unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2009-164968, filed Jul. 13, 2009; theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to a flow line recognitionsystem configured to recognize paths of people as flow lines.

BACKGROUND

Systems have been developed that recognize paths of customers who movein stores, such as supermarkets, convenience stores, etc.Conventionally, such systems include those that use camera images torecognize flow lines of customers (moving objects), those that use radiotags to recognize flow lines, and those that uses multi-laser scannersfor the flow line recognition.

The flow line recognition system using camera images requires manycameras in order to cover a monitoring area entirely. Therefore, thenumber of cameras increases if the monitoring area becomes wider. If thecameras increase, the following problems occur.

(1) The capacity of storage devices for recording captured image data isinevitably large.

(2) The cameras require complicated control.

(3) The maintenance of the cameras becomes more costly.

Thus, the flow line recognition system using camera images is unsuitablefor stores with a large sales floor space.

Since the flow line recognition system using radio tags does not useimage data, it can comprise a small storage device. In order toaccurately detect the positions of customers as moving objects, however,the number of radio tag readers needs to be increased. If the tagreaders increase, radio-wave interference is caused between them, sothat data on the radio tags cannot be read. Thus, the number of radiotag readers that can be installed within the monitoring area isphysically limited. Consequently, the accuracy of this system is lowerthan that of the system that uses camera images.

According to this system, moreover, the radio tags need to be carried bythe customers. To this end, a complicated operation is required suchthat, for example, one radio tag is affixed to each shopping cart.

According to the flow line recognition system using multi-laserscanners, on the other hand, it is unnecessary for anything to becarried by customers. However, this system detects customers (movingobjects) by scanning with laser lights by means of the laser scanners.If the customers overlap one another in the direction of emission of thelaser lights, those on the far side cannot be detected, although thoseon the near side can. Consequently, data having so far been used totrack the far-side customers will disappear. Thus, this system isunsuitable for use in a busy monitoring area.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of a store to which one embodiment is applied;

FIG. 2 is a diagram used for illustrating a laser light projected from alaser scanner according to the same embodiment;

FIG. 3 is a block diagram showing one embodiment of a flow linerecognition system;

FIG. 4 is a structural diagram showing data to be stored in a laser flowline database according to the same embodiment;

FIG. 5 is a structural diagram showing data to be stored in a cameraflow line database according to the same embodiment;

FIG. 6 is a diagram used for illustrating a specific area according tothe same embodiment;

FIG. 7 is a structural diagram showing data to be stored in a specificarea database according to the same embodiment;

FIG. 8 is a structural diagram showing data to be stored in an extractedflow line database according to the same embodiment;

FIG. 9 is a structural diagram showing a final flow line file stored ina final flow line database according to the same embodiment;

FIG. 10 is a flowchart showing a processing procedure of a determinationunit according to the same embodiment;

FIG. 11 is a flowchart showing a processing procedure of a final flowline generation unit according to the same embodiment; and

FIG. 12 is a flowchart showing a processing procedure of a flow linecoupling unit according to the same embodiment.

DETAILED DESCRIPTION

In general, according to one embodiment, the flow line recognitionsystem includes a first device, a first recording unit, a second device,a second recording unit, a generation unit. The first device detects aposition of a moving object in a monitoring area by laser scanning. Thefirst recording unit records position information indicative of theposition of the moving object detected by the first device as movingobject information for the monitoring area, along with time informationindicative of a time when the moving object is in the detected position.The second device detects a position of the moving object in a specificarea, which is a part of the monitoring area, more accurately than thefirst device. The second recording unit records position informationindicative of the position of the moving object detected by the seconddevice as moving object information for the specific area, along withtime information indicative of a time when the moving object is in thedetected position. The generation unit generates flow line informationindicative of a path of the moving object moving in the monitoring area,based on the moving object information for the monitoring area recordedin the first recording unit, and the moving object information for thespecific area recorded in the second recording unit.

FIG. 1 is a plan view of a store 1 in which the flow line recognitionsystem of the present embodiment is constructed. In the store 1,commodity racks 2, checkout counters 3, and doorways 4 are located inpredetermined positions. Each checkout counter 3 comprises apoint-of-sales terminal and barcode scanner, which are individually usedfor accounting services.

In the present embodiment, the whole area of the store 1 except a blockwhere the commodity racks 2 and checkout counters 3 are installed isdefined as a monitoring area. Of the monitoring area, a hatched area 5,that is, an area on the side where the checkout counters 3 are accessed,as shown in FIG. 1, is defined as a specific area. The flow linerecognition system is constructed so that it can recognize, as a flowline, a path along which a customer having entered the store through anyof the doorways 4 moves through the monitoring area and specific areaand leaves the store through any of the doorways 4.

This system uses a multi-laser scanner model and camera image model incombination. Specifically, the camera image model is adopted for thespecified area 5, and the multi-laser scanner model for the monitoringarea, that is, the whole area except the specific area 5.

In the monitoring area for which the multi-laser scanner model isadopted, a plurality of (12 in FIG. 1) laser scanners 6 (6A to 6L) arelocated individually in the positions shown in FIG. 1. Around thespecific area 5 for which the camera image model is adopted, a pluralityof (4 in FIG. 1) cameras 7 (7A to 7D) are located individually in thepositions shown in FIG. 1.

FIG. 3 is a block diagram showing the principal configuration of theflow line recognition system. This system comprises a laser scannercontrol unit 21, camera control unit 22, laser flow line generation unit23, camera flow line generation unit 24, laser distance image database25, camera image database 26, laser flow line database 27, and cameraflow line database 28.

The laser scanner control unit 21 controls a scanning operation of eachlaser scanner 6.

As shown in FIG. 2, each laser scanner 6 scans the monitoring area witha laser light parallel to the floor surface so as to cover a wide angleof 180° at a height of 10 to 20 cm above floor level. The laser lightemitted from the laser scanner 6 is reflected when it hits obstacles,such as stationary and moving objects. The reflected light returns tothe laser scanner 6 as a projected light source.

The control unit 21 counts the time required for the return of thereflected light after the laser-light scanning by each laser scanner 6,in order to calculate a distance from each scanner 6 to the obstacle.Based on calculated values of distances from the obstacle in allscanning directions, the control unit 21 synthesizes a laser distanceimage for each laser scanner 6. The laser distance image containsinformation on the stationary and moving objects on a horizontal sectioncorresponding to the height of projection of the laser light.

The control unit 21 records the laser distance image obtained for eachlaser scanner 6 in the laser distance image database 25. In doing this,the control unit 21 tags each laser distance image with informationindicative of the laser scanner 6 from which the image is obtained andinformation indicative of the start time of scanning.

The laser flow line generation unit 23 generates laser flow lineinformation based on data recorded in the laser distance image database25.

Each laser scanner 6 constantly scans the same range in the sameposition. Thus, the laser length to stationary objects, such as walls orcommodity racks, cannot change beyond the range of system errors.

The generation unit 23 analyzes laser distance images obtained from oneand the same laser scanner 6 in a time series for individual pixels.Those pixels which have not changed for a predetermined time (e.g., 10seconds) or more are detected as a background image.

If the background image is detected, its difference is obtained fromeach laser distance image of the same laser scanner 6 in ascending orderof the start time of scanning. The generation unit 23 can extract themoving object by obtaining the difference of the background image. If anew moving object is extracted, the generation unit 23 issues a flowline ID (laser flow line ID) appropriate to this moving object.

The generation unit 23 tracks the extracted moving object. Then, theposition of the moving object being tracked is detected in a fixedperiod. The generation unit 23 functions as a detection unit, whichrecognizes the moving object, based on the distance information obtainedduring the time interval required for the return of the reflected lightafter the laser-light scanning by the laser scanners 6A to 6L, anddetects the position of the moving object.

The generation unit 23 converts the position of the moving object intoorthogonal coordinates (X, Y) in the monitoring area. In the presentembodiment, the lower left corner of the store 1 shown in FIG. 1 isassumed to be the origin O (0, 0) of the orthogonal coordinates (X, Y).

The generation unit 23 generates the laser flow line information everytime the position of the moving object being tracked is detected. Thelaser flow line information contains the values of the orthogonalcoordinates (X, Y) indicative of the position of the moving object,laser flow line ID issued for the moving object, and detection timeinformation on the moving object. The detection time information isindicative of the start time of scanning of the laser distance imagefrom which the moving object is extracted for the first time. Thegeneration unit 23 records the laser flow line information in the laserflow line database 27.

FIG. 4 is a structural diagram showing data to be stored in the laserflow line database 27. The database 27 stores the laser flow lineinformation containing the laser flow line ID, orthogonal coordinates,and detection time in ascending order of the detection time. The laserflow line information is tagged with a processing flag. The processingflag is “0” at the point in time when the corresponding laser flow lineinformation is recorded in the laser flow line database 27. Theprocessing flag becomes “1” when it is processed in a final flow linegeneration unit 41, which will be described later. Alternatively, theprocessing flag may be set so that it is “1” when it is recorded in thelaser flow line database 27 and becomes “0” when processed.

The camera control unit 22 controls the image capture operation of eachcamera 7. Each camera 7 captures images of customers moving in thespecific area 5 from each corresponding position shown in FIG. 1. Thecamera control unit 22 fetches image data captured by each camera andrecords it in the camera image database 26. In doing this, the cameracontrol unit 22 tags each image data with information indicative of thecamera used to capture the image data and information on the time ofimage capture.

The camera flow line generation unit 24 generates camera flow lineinformation based on the camera image data recorded in the camera imagedatabase 26.

The camera flow line generation unit 24 extracts the moving object byprocessing the image data, captured by the cameras 7A to 7D, based onthe known volume intersection method. If a new moving object isextracted, the generation unit 24 issues a flow line ID (camera flowline ID) appropriate to this moving object.

The generation unit 24 tracks the extracted moving object. Then, theposition of the moving object being tracked is detected in a fixedperiod. The generation unit 24 functions as a detection unit, whichrecognizes the moving object, based on the image information captured bythe cameras 7, and detects the position of the moving object.

The generation unit 24 converts the position of the moving object intoorthogonal coordinates (X, Y) in the monitoring area.

The generation unit 24 generates the camera flow line information everytime the position of the moving object being tracked is detected. Thecamera flow line information contains the values of the orthogonalcoordinates (X, Y) indicative of the position of the moving object,camera flow line ID issued for the moving object, and detection timeinformation on the moving object. The detection time information isindicative of the time of capture of the image from which the movingobject is extracted for the first time. The generation unit 24 recordsthe camera flow line information in the camera flow line database 28.

FIG. 5 is a structural diagram showing data to be stored in the cameraflow line database 28. The database 28 stores the camera flow lineinformation containing the camera flow line ID, orthogonal coordinates,and detection time in ascending order of the detection time.

The flow line recognition system using camera images recognizes movingobjects by processing image data based on an image of the specific area5 captured from above. Thus, the probability of losing track of themoving objects is lower than in the case of a flow line recognitionsystem using multi-laser scanners that recognize moving objects byprocessing laser distance images on a horizontal section. Thus, the flowline recognition system using camera images can detect flow lines ofmoving objects more accurately than the flow line recognition systemthat uses multi-laser scanners.

A first device 100 that detects the position of each moving object inthe monitoring area by laser scanning comprises the laser scanners 6,laser scanner control unit 21, and laser flow line generation unit 23. Asecond device 200 that detects the position of each moving object moreaccurately than the first device 100 comprises the cameras 7A to 7D,camera control unit 22, and camera flow line generation unit 24.

Returning to FIG. 3, the flow line recognition system further comprisesa determination unit 31, specific area database 32, and extracted flowline database 33.

As shown in FIG. 6, the specific area 5 is rectangular in the orthogonalcoordinate system. Therefore, if the orthogonal coordinates of points P1and P2 at the upper left end and lower right end with respect to theorigin O (0, 0) are given by (x1, y2) and (x2, y1), respectively, thespecific area 5 is a set of point coordinates based on an X-coordinaterange of x1 to x2 and Y-coordinate range of y1 to y2.

As shown in FIG. 7, the specific area database 32 stores the values x1to x2 representative of the X-coordinate range and values y1 to y2representative of the Y-coordinate range, corresponding to an area IDfor the identification of the specific area 5.

The determination unit 31 fetches each item of laser flow lineinformation recorded in the laser flow line database 27. Referring thento data on the coordinate values set in the specific area database 32,the determination unit 31 determines whether or not a moving object hasjust entered the specific area 5. If laser flow line information at thepoint in time when the specific area 5 is entered by the moving objectis detected, the determination unit 31 records the laser flow lineinformation as entry flow line information in the extracted flow linedatabase 33.

Referring to the data on the coordinate values set in the specific areadatabase 32, moreover, the determination unit 31 determines whether ornot the moving object has just exited the specific area 5. If laser flowline information at the point in time when the specific area 5 isentered by the moving object is detected, the determination unit 31records the laser flow line information as exit flow line information tothe extracted flow line database 33.

FIG. 8 is a structural diagram showing data to be stored in theextracted flow line database 33. The database 33 stores the laser flowline information containing the laser flow line ID, orthogonalcoordinates, and detection time in the ascending order of the detectiontime. The laser flow line information is tagged with a determinationflag. The determination flag is “1” when the laser flow line informationis the entry flow line information, and “0” for the exit flow lineinformation. Alternatively, the determination flag may be set so thatfor the entry flow line information is “0” and that for the exit flowline information is “1”.

The determination unit 31 executes processing of a procedure shown inthe flowchart of FIG. 10 every time the laser flow line information iswritten to the laser flow line database 27.

First, the determination unit 31 acquires laser flow line information Lkwritten in the database 27 (Act 1). The determination unit 31 retrievesthe specific area database 32 with orthogonal coordinates (Xk, Yk) ofthe laser flow line information Lk (Act 2). Then, the determination unit31 determines whether or not a position represented by the orthogonalcoordinates (Xk, Yk) is within the specific area 5 (Act 3). If the X-and Y-coordinates Xk and Yk are “x1<Xk≦x2” and “y1≦Yk≦y2”, respectively,the laser flow line information L is data on a moving object locatedwithin the specific area 5. If the X-coordinate is not “x1≦Xk≦x2” or ifthe Y-coordinate is not “y1≦Yk≦y2”, the laser flow line information

Lk is data on a moving object located outside the specific area 5.

If the position represented by the orthogonal coordinates (Xk, Yk) isdetermined to be outside the specific area 5 (NO in Act 3), thedetermination unit 31 retrieves data from the extracted flow linedatabase 33 with the laser flow line ID of the laser flow lineinformation Lk, in descending order of the detection time (Act 4). Then,the presence of laser flow line information Lm containing the same flowline ID is determined (Act 5).

If the laser flow line information Lm is not detected (NO in Act 5), thelaser flow line information Lk is data on a moving object having notentered the specific area 5. In this case, the determination unit 31terminates processing for the laser flow line information Lk.

If the laser flow line information Lm is detected (YES in Act 5), thedetermination unit 31 checks a determination flag Fm of the laser flowline information Lm (Act 6). If the determination flag Fm is “0” (NO inAct 6), the laser flow line information

Lk is data on a moving object determined to have exited the specificarea 5 by a previous determination process. In this case, thedetermination unit 31 terminates the processing for the laser flow lineinformation Lk.

If the determination flag Fm is “1” (YES in Act 6), the laser flow lineinformation Lk is data on a moving object determined to have not exitedthe specific area 5 by the previous determination process, that is, amoving object having just exited the specific area 5. In this case, thedetermination unit 31 stores the laser flow line information Lk in theextracted flow line database 33 (Act 7). In doing this, thedetermination unit 31 sets the determination flag Fm of the laser flowline information Lk to “0” (Act 8). Thereupon, the determination unit 31terminates the processing for the laser flow line information Lk.

If the position represented by the orthogonal coordinates (Xk, Yk) isdetermined to be within the specific area 5 (YES in Act 3), thedetermination unit 31 retrieves the data from the extracted flow linedatabase 33 with the laser flow line ID of the laser flow lineinformation Lk, in descending order of the detection time (Act 9). Then,the presence of the laser flow line information Lm containing the sameflow line ID is determined (Act 10).

If the laser flow line information Lm is not detected (NO in Act 10),the laser flow line information Lk is data on a moving object determinedto have not entered the specific area 5 by the previous determinationprocess, that is, a moving object having just entered the specific area5. In this case, the determination unit 31 stores the laser flow lineinformation Lk in the extracted flow line database 33 (Act 12). In doingthis, the determination unit 31 sets the determination flag Fm of thelaser flow line information Lk to “1” (Act 13). Thereupon, thedetermination unit 31 terminates the processing for the laser flow lineinformation Lk.

If the laser flow line information Lm is detected (YES in Act 10), thedetermination unit 31 checks the determination flag Fm of the laser flowline information Lm (Act 11). If the determination flag Fm is not “1”(NO in Act 11), the laser flow line information Lk is data on a movingobject having just entered the specific area 5 again after having onceexited. In this case, the determination unit 31 stores the laser flowline information Lk in the extracted flow line database 33 (Act 12). Indoing this, the determination unit 31 sets the determination flag Fm ofthe laser flow line information Lk to “1” (Act 13). Thereupon, thedetermination unit 31 terminates the processing for the laser flow lineinformation Lk.

If the determination flag Fm is “1” (YES in Act 11), the laser flow lineinformation Lk is data on a moving object determined to have entered thespecific area 5 by the previous determination process. In this case, thedetermination unit 31 terminates the processing for the laser flow lineinformation Lk.

As the determination unit 31 executes the processes of Acts 1 to 13, thelaser flow line information with the determination flag Fm at “1” andlaser flow line information with the determination flag Fm at “0” arerecorded in a time series in the extracted flow line database 33. Thelaser flow line information with the determination flag Fm at “1” is thelaser flow line information Lk obtained when the specific area 5 isentered by the moving object. The laser flow line information with thedetermination flag Fm at “0” is the laser flow line information Lkobtained when the specific area 5 is exited by the moving object.

Returning to FIG. 3, the flow line recognition system further comprisesthe final flow line generation unit 41, a flow line coupling unit 42, aflow line reproduction unit 43, a final flow line database 44, and adisplay unit 45. The display unit 45 is a color display, for example.

The final flow line generation unit 41 generates final flow line datafor each customer based on processing of a procedure shown in theflowchart of FIG. 11. A program is automatically started to execute thisprocessing at a predetermined time. Alternatively, the program isstarted to execute the processing when a predetermined command is inputby the operation of an input device, such as a keyboard.

First, the generation unit 41 retrieves the laser flow line informationstored in the laser flow line database 27 in ascending order of thedetection time (Act 21). Then, the presence of laser flow lineinformation Ln containing a processing flag F at “0” is determined (Act22).

If the laser flow line information Ln is detected (YES in Act 22), thegeneration unit 41 further retrieves the laser flow line database 27with the flow line ID (outstanding flow line ID) of the laser flow lineinformation Ln. Of the laser flow line information containing the sameflow line ID as the outstanding flow line ID, all items of laser flowline information Lnn with the processing flag F at “0” are extracted inascending order of the detection time (Act 23). The generation unit 41updates a processing flag Fn of all the extracted items of laser flowline information Lnn to “1” (Act 24).

The generation unit 41 generates a final flow line file 50 of the datastructure shown in FIG. 9. Then, the orthogonal coordinates of all theitems of laser flow line information Lnn and detection time data arestored in the final flow line file 50 in ascending order of thedetection time (Act 25).

The generation unit 41 generates a new customer ID. This customer ID issaved as a filename in the final flow line database 44 (Act 26).

The generation unit 41 generates a processing request command C1. Thecommand C1 contains the new customer ID, outstanding flow line ID, anddetection time of the laser flow line information Ln. The generationunit 41 transmits the command C1 to the flow line coupling unit 42 (Act27).

On receipt of the command C1 from the generation unit 41, the couplingunit 42 couples the entry or exit flow line information to the cameraflow line information based on the procedure shown in the flowchart ofFIG. 12.

First, the coupling unit 42 retrieves data from the extracted flow linedatabase 33 in ascending order of the detection time (Act 31). Then, thepresence of laser flow line information Li that agrees with theoutstanding flow line ID contained in the command C1 is determined (Act32).

If the laser flow line information Li is detected (YES in Act 32), thecoupling unit 42 determines whether or not a detection time Ti of thelaser flow line information Li is preceded by a detection time Tccontained in the command C1 (Act 33).

If the detection time Ti is earlier than the detection time Tc (NO inAct 33), the laser flow line information Li is data on a customer havingmade a purchase earlier than a customer to whom the customer IDcontained in the command C1 is allocated. In this case, the couplingunit 42 continues to retrieve the extracted flow line database 33 (Act31).

If the detection time Ti is later than the detection time Ti (YES in Act33), the laser flow line information Li is data on the customer to whomthe customer ID contained in the command C1 is allocated. In this case,the coupling unit 42 checks out a determination flag Fi of the laserflow line information Li (Act 34).

If the determination flag Fi is “0” (NO in Act 34), the laser flow lineinformation Li is data at the time of exit of the moving object from thespecific area 5. In this case, the coupling unit 42 continues toretrieve the extracted flow line database 33 (Act 31).

If the determination flag Fi is “1” (YES in Act 34), the laser flow lineinformation Li is data at the time of entry of the moving object intothe specific area 5. In this case, the coupling unit 42 acquires data onorthogonal coordinates (Xi, Yi) and the detection time Ti from the laserflow line information Li (Act 35).

The coupling unit 42 retrieves the camera flow line database 28. Then,it extracts one item of camera flow line information Rj of which thedetection time is nearest to the detection time Ti (Act 36).

The coupling unit 42 acquires orthogonal coordinates (Xj, Yj) of thecamera flow line information Rj (Act 37). Then, it calculates a distanced from a point specified by the orthogonal coordinates (Xj, Yj) to apoint specified by the orthogonal coordinates (Xi, Yi) (Act 38).

The coupling unit 42 determines whether or not the distance d is lessthan a predetermined threshold D (Act 39). The threshold D is set to onemeter or less, e.g., 80 cm. If the distance d is less than the thresholdD (YES in Act 39), the camera flow line information Rj and laser flowline information Li are regarded as items of flow line information onone and the same customer.

In this case, the coupling unit 42 acquires the camera flow line ID fromthe camera flow line information Rj (Act 40). Then, it retrieves thecamera flow line database 28 and extracts all items of camera flow lineinformation Rjj containing the same camera flow line ID (Act 41).

The coupling unit 42 opens the final flow line file 50, of which thefilename is given by the customer ID contained in the command C1,through the final flow line database 44. Then, the coupling unit 42sequentially writes to the file 50 the orthogonal coordinates and dataon the detection times of all items of camera flow line information Rjjextracted from the camera flow line database 28 (Act 42).

When the data writing is finished, the coupling unit 42 continues toretrieve the extracted flow line database 33 (Act 31).

If the distance d is not less than the threshold D (NO in Act 39), thecamera flow line information Rj and laser flow line information Li areregarded as items of flow line information on another customer. In thiscase, the coupling unit 42 calculates a time difference t between adetection time Tj of the camera flow line information Rj and thedetection time Ti of the laser flow line information Li (Act 43).

The coupling unit 42 determines whether or not the time difference t isgreater than a predetermined threshold T. The threshold T is set to oneminute or less, e.g., 30 seconds (Act 44).

If the time difference t is not greater than the predetermined value T(NO in Act 43), the coupling unit 42 continues to retrieve the cameraflow line database 28 (Act 36). Then, the coupling unit 42 extracts theinformation Rj of which the detection time Tj is nearest to thedetection time Ti, out of unextracted camera flow line information. Ifthe camera flow line information Rj is extracted, the processes of Acts37 to 44 are executed again.

If the time difference t exceeds the threshold T

(YES in Act 43), the coupling unit 42 returns to the process for theretrieval of the extracted flow line database 33 (Act 31).

If the retrieval of the extracted flow line database 33 is finished (NOin Act 32), the coupling unit 42 outputs a response command C2 for thetermination of processing to the final flow line generation unit 41 (Act45). The command C2 contains the customer ID in the processing requestcommand C1.

The generation unit 41 having output the processing request command C1in Act 27 of FIG. 11 waits for the response command C2 for thetermination of processing (Act 28). On receipt of the command C2 fromthe coupling unit 42 (YES in Act 28), the generation unit 41 opens thefinal flow line file 50, of which the filename is given by the customerID contained in the command C1, through the final flow line database 44.Then, the generation unit 41 rearranges the items of final flow lineinformation (orthogonal coordinates and detection time) stored in thefile 50 in ascending order of the detection time (Act 29).

Thus, the final flow line information stored in the final flow line file50 is a combination of flow line information for the pursuit of a pathof a customer moving in the specific area 5 and flow line informationfor the pursuit of a path of the same customer moving in the monitoringarea exclusive of the specific area 5.

The generation unit 41 and flow line coupling unit 42 constitute ageneration unit 300, which generates flow line information indicative ofa path of a moving object moving in the store 1, based on items ofmoving object information recorded in the laser flow line database 27and camera flow line database 28. The coupling unit 42 comprises acomparison unit 301 that compares respective position information andtime information of the in-store moving object information andin-specific-area moving object information.

The flow line reproduction unit 43 causes the display unit 45 to displaya screen that indicates a layout of the store 1. Then, a flow line ofthe customer moving in the store 1 is reproduced on the screen, based onthe final flow line information stored in the final flow line file 50.

When this is done, a flow line recognized by the flow line recognitionsystem using multi-laser scanners is displayed in the monitoring areaexclusive of the specific area 5. On the other hand, a flow linerecognized by the flow line recognition system using camera images isdisplayed in the specific area 5.

In a large supermarket, an area in front of the checkout counters 3 isoften jammed with customers waiting to pay or moving around, shopping.

According to the flow line recognition system of the present embodiment,the specific area 5 is located in front of the checkout counters 3. Theflow line recognition system using camera images is adopted for thespecific area 5. This system can detect moving objects more accuratelythan the flow line recognition system using multi-laser scanners.Accordingly, the flow line recognition system of the present embodimenthas an effect that it can highly accurately recognize customer flowlines even in the jammed specific area 5.

On the other hand, the flow line recognition system using multi-laserscanners is adopted for the area other than the specific area 5.Although the accuracy of this system is lower than that of the flow linerecognition system of the camera-image type, the construction andmaintenance of the system are economical, that is, have low costs. Thelarger the monitoring area, as in a large supermarket, the greater thisadvantage is.

This invention is not limited directly to the embodiment describedabove, and in carrying out the invention, its constituent elements maybe embodied in modified forms without departing from the spirit or scopeof the invention.

In the embodiment described above, the flow line recognition systemusing camera images is used as the second device 200. However, thesecond device 200 is not limited to the flow line recognition system ofthat type. In short, it is necessary only that the second device 200 beable to detect the position of each moving object more accurately thanin the flow line recognition system that uses multi-laser scanners forthe first device 100.

In the embodiment described above, the specific area 5 is located infront of the checkout counters 3. However, the specific area 5 is notlimited to this location. Further, the specific area is not limited to arectangular shape.

In the embodiment, a plurality of specific areas 5 may be arranged inthe monitoring area. In this case, the cameras 7A to 7D, camera controlunit 22, camera image database 26, camera flow line generation unit 24,and camera flow line database 28 are provided as a systematic group foreach of the specific areas. The determination unit 31 tags laser flowline information for the entry into or exit from each specific area withan area ID for the specific area 5 concerned. The coupling unit 42couples the laser flow line information to the camera flow lineinformation with reference to the camera flow line database 28 of thegroup identified by the area ID of the laser flow line information.

In the embodiment described above, items of flow line information on thesame moving object are coupled by matching the laser flow lineinformation and camera flow line information extracted into theextracted flow line database 33. However, the method of coupling theitems of flow line information is not limited to this. For example, theitems of flow line information on the same moving object may be coupledby matching the laser flow line information and camera flow lineinformation in the laser flow line database 27.

However, the laser flow line information extracted into the extractedflow line database 33 is information for the entry into or exit from thespecific area 5.

Thus, the number of necessary items of laser flow line information forthe determination of coupling can be considerably reduced by using themethod of the present embodiment. As a result, the processing loadrequired by the flow line coupling unit 42 can be considerably reduced.

In the embodiment described above, the laser flow line information andcamera flow line information are coupled on condition that the spacebetween each two adjacent positions is not greater than a predeterminedvalue and the time difference is minimal. However, the couplingcondition is not limited to this and may be combined with otherconditions.

In the embodiment described above, the determination unit 31 isconfigured to detect the flow line information for the entry or exit ofeach moving object into or from the specific area 5. However, the cameraflow line information recorded in the camera flow line database 28, likethe laser flow line information, contains the orthogonal coordinates (X,Y) and detection time data. As in the case of the above-describedembodiment, therefore, the determination unit 31 may be configured todetect the flow line information for the entry or exit of each movingobject into or from the specific area 5 from the camera flow lineinformation and record it to the extracted flow line database 33.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel methods and systems describedherein may be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the methods andsystems described herein may be made without departing from the spiritof the inventions. The accompanying claims and their equivalents areintended to cover such forms or modifications as would fall within thescope and spirit of the inventions.

1. A flow line recognition system comprising: a first device whichdetects a position of a moving object in a monitoring area by laserscanning; a first recording unit which records position informationindicative of the position of the moving object detected by the firstdevice as moving object information for the monitoring area, along withtime information indicative of a time when the moving object is in thedetected position; a second device which detects a position of themoving object in a specific area, which is a part of the monitoringarea, more accurately than the first device; a second recording unitwhich records position information indicative of the position of themoving object detected by the second device as moving object informationfor the specific area, along with time information indicative of a timewhen the moving object is in the detected position; and a generationunit which generates flow line information indicative of a path of themoving object moving in the monitoring area, based on the moving objectinformation for the monitoring area recorded in the first recordingunit, and the moving object information for the specific area recordedin the second recording unit.
 2. The system of claim 1, wherein thefirst device comprises a laser scanner which scans the monitoring areawith a laser light parallel to a floor surface and a detection unitwhich recognizes the moving object, based on distance informationobtained during a time interval required for the return of a reflectedlight beam after the laser-light scanning by the laser scanner, anddetects a position of the moving object.
 3. The system of claim 1,wherein the second device comprises a camera which captures an image ofthe specific area and a detection unit which recognizes the movingobject, based on image information captured by the camera, and detects aposition of the moving object.
 4. The system of claim 2, wherein thesecond device comprises a camera which captures an image of the specificarea and a detection unit which recognizes the moving object, based onimage information captured by the camera, and detects a position of themoving object.
 5. The system of claim 1, wherein the generation unitcomprises a comparison unit which compares respective positioninformation and time information of the moving object information forthe monitoring area and the moving object information for the specificarea and a coupling unit which couples, as information on one and thesame moving object, the moving object information for the monitoringarea and the moving object information for the specific area betweenwhich the difference in position information is not greater than apredetermined value and the difference in time information is thesmallest.
 6. The system of claim 5, further comprising a determinationunit which determines whether or not the specific area is entered by themoving object, based on the position information on the moving objectdetected by the first device, wherein the comparison unit compares themoving object information for the monitoring area, on the moving objectdetermined to have entered the specific area by the determination unit,with the moving object information for the specific area.
 7. The systemof claim 6, wherein the first device comprises a laser scanner whichscans the monitoring area with a laser light parallel to a floor surfaceand a detection unit which recognizes the moving object, based ondistance information obtained during a time interval required for thereturn of a reflected light beam after the laser-light scanning by thelaser scanner, and detects a position of the moving object.
 8. Thesystem of claim 5, further comprising a determination unit whichdetermines whether or not the specific area is entered by the movingobject, based on the position information on the moving object detectedby the second device, wherein the comparison unit compares the movingobject information for the monitoring area, on the moving objectdetermined to have entered the specific area by the determination unit,with the moving object information for the specific area.
 9. The systemof claim 8, wherein the second device comprises a camera which capturesan image of the specific area and a detection unit which recognizes themoving object, based on image information captured by the camera, anddetects a position of the moving object.
 10. The system of claim 5,further comprising a determination unit which determines whether or notthe specific area is exited by the moving object, based on the positioninformation on the moving object detected by the first device, whereinthe comparison unit compares the moving object information for themonitoring area, on the moving object determined to have exited thespecific area by the determination unit with the moving objectinformation for the specific area.
 11. The system of claim 10, whereinthe first device comprises a laser scanner which scans the monitoringarea with a laser light parallel to a floor surface and a detection unitwhich recognizes the moving object, based on distance informationobtained during a time interval required for the return of a reflectedlight beam after the laser-light scanning by the laser scanner, anddetects a position of the moving object.
 12. The system of claim 5,further comprising a determination unit which determines whether or notthe specific area is exited by the moving object, based on the positioninformation on the moving object detected by the second device, whereinthe comparison unit compares the moving object information for themonitoring area, on the moving object determined to have exited thespecific area by the determination unit, with the moving objectinformation for the specific area.
 13. The system of claim 12, whereinthe second device comprises a camera which captures an image of thespecific area and a detection unit which recognizes the moving object,based on image information captured by the camera, and detects aposition of the moving object.
 14. A flow line recognition systemcomprising: a laser scanner which scans a monitoring area with a laserlight parallel to a floor surface; a first detection unit whichrecognizes a moving object, based on distance information obtainedduring a time interval required for the return of a reflected light beamafter the laser-light scanning by the laser scanner, and detects aposition of the moving object; a first recording unit which recordsposition information indicative of the position of the moving objectdetected by the first detection unit as moving object information forthe monitoring area, along with time information indicative of a timewhen the moving object is in the detected position; a camera whichcaptures an image of a specific area, which is a part of the monitoringarea; a second detection unit which recognizes the moving object, basedon image information captured by the camera, and detects a position ofthe moving object; a second recording unit which records positioninformation indicative of the position of the moving object detected bythe second detection unit as moving object information for the specificarea, along with time information indicative of a time when the movingobject is in the detected position; and a generation unit whichgenerates flow line information indicative of a path of the movingobject moving in the monitoring area, based on the moving objectinformation for the monitoring area recorded in the first recordingunit, and the moving object information for the specific area recordedin the second recording unit.
 15. The system of claim 14, wherein thegeneration unit comprises a comparison unit which compares respectiveposition information and time information of the moving objectinformation for the monitoring area and the moving object informationfor the specific area and a coupling unit which couples, as informationon one and the same moving object, the moving object information for themonitoring area and the moving object information for the specific areabetween which the difference in position information is not greater thana predetermined value and the difference in time information is thesmallest.
 16. The system of claim 15, further comprising a determinationunit which determines whether or not the specific area is entered by themoving object, based on the position information on the moving objectdetected by the first detection unit, wherein the comparison unitcompares the moving object information for the monitoring area, on themoving object determined to have entered the specific area by thedetermination unit, with the moving object information for the specificarea.
 17. The system of claim 15, further comprising a determinationunit which determines whether or not the specific area is entered by themoving object, based on the position information on the moving objectdetected by the second detection unit, wherein the comparison unitcompares the moving object information for the monitoring area, on themoving object determined to have entered the specific area by thedetermination unit, with the moving object information for the specificarea.
 18. The system of claim 15, further comprising a determinationunit which determines whether or not the specific area is exited by themoving object, based on the position information on the moving objectdetected by the first detection unit, wherein the comparison unitcompares the moving object information for the monitoring area, on themoving object determined to have exited the specific area by thedetermination unit, with the moving object information for the specificarea.
 19. The system of claim 15, further comprising a determinationunit which determines whether or not the specific area is exited by themoving object, based on the position information on the moving objectdetected by the second detection unit, wherein the comparison unitcompares the moving object information for the monitoring area, on themoving object determined to have exited the specific area by thedetermination unit, with the moving object information for the specificarea.